The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Canonical URLs and screen scraping

    Canonical URLs and screen scraping

    Technical SEO Issues
    5 3 1.2k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • friendlymachine
      friendlymachine last edited by

      So a little question here. I was looking into a module to help implement canonical URLs on a certain CMS and I came a cross a snarky comment about relative vs. absolute URLs being used. This person was insistent that relative URLs are fine and absolute URLs are only for people who don't know what they are doing.

      My question is, if using relative URLs, doesn't it make it easier to have your content scraped? After all, if you do get your content scraped at least it would point back to your site if using absolute URLs, right? Am I missing something or is my thinking OK on this?

      Any feedback is much appreciated!

      1 Reply Last reply Reply Quote 0
      • RobertFisher
        RobertFisher last edited by

        John

        You can use either and the web is full of those who go back and forth on this issue. My guess is that any really good scraper software can likely deal with absolute urls today. The advantage that we like with relative is all about page load speed - the file size is smaller with relative urls.

        So, you will get arguments both ways. If scraping is a huge issue for you, maybe you go with absolute. We know people will scrape content and we continue with relative for the above reason and because it is easier to make certain changes/linking/redirects within a CMS.

        Oh as to people who use absolutes not knowing what they are doing....that is bunk. They have other priorities, maybe.

        friendlymachine 1 Reply Last reply Reply Quote 2
        • AlanMosley
          AlanMosley last edited by

          People don’t abuse people when you have facts on their side, reminds me of "you don’t believe in global warming, because your un-educated" argument.
          I have seen just in the last few weeks where using absolute url has got me a link. I wrote a youmoz article with a link to my website, it has been copied and has the link in it. Of cause being on SEOMoz, I have to use a absolute url back to myself
          I don’t usually use absolute links on my own site, I think search engines almost always know who copied who.
          I agree with rob, but I will add, a good screen scraper will remove a canonical tag, but removing absolute links is not so easy, as you then have broken links, also I believe if you have image in the article linking back to you, search engines will know who the real owner is, same with css, js and a number of other refs. Screen scrapers rarely get credit for these reasons as well as the fact that if your site has a lot of duplicate, then it is obvious that you are the one coping It’s either the one site is copied from many locations or many locations have copied from the one site.

          friendlymachine 1 Reply Last reply Reply Quote 2
          • friendlymachine
            friendlymachine @RobertFisher last edited by

            Thanks, Robert. Your rational for using relative links make sense. I appreciate you helping me sort through the noise on this issue.

            John

            1 Reply Last reply Reply Quote 0
            • friendlymachine
              friendlymachine @AlanMosley last edited by

              Thanks for your reply, Alan. I also considered a screen scraper removing the canonical tag, but to me screen scraping seemed lazy in the first place and so maybe they wouldn't bother in most cases. I guess that a best practice with canonicals is really situation dependent.

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • If a URL canonically points to another link, is that URL indexed?
                Coolguyry
                Coolguyry
                0
                4
                120

              • When to use canonical urls
                Dr-Pete
                Dr-Pete
                0
                5
                247

              • How to find original URLS after Hosting Company added canonical URLs, URL rewrites and duplicate content.
                Nobody1560986989723
                Nobody1560986989723
                0
                2
                366

              • Canonical URL Issue
                influxmedia
                influxmedia
                0
                4
                376

              • Canonical URL
                AlanMosley
                AlanMosley
                0
                2
                483

              • Trailing Slashes In Url use Canonical Url or 301 Redirect?
                RyanKent
                RyanKent
                0
                7
                2.7k

              • Blank Canonical URL
                OptimizeSmart
                OptimizeSmart
                0
                2
                783

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy