The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Web Design
    4. How to command Robots.txt to this:

    How to command Robots.txt to this:

    Web Design
    7 3 151
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Kokolo
      Kokolo last edited by

      Hi,

      So for some reason I have this unexplained issues in webmaster tools. Check them out:

      http://prntscr.com/7n1nj8

      See that iSeeCars.com? How to remove it? Is it just disallow: iseecars.com?

      Or should I disallow the search to be crawled?

      Regards,

      1 Reply Last reply Reply Quote 0
      • Andy.Drinkwater
        Andy.Drinkwater last edited by

        Is this a page / article in your site anywhere Nikola? I'm wondering if it's some kind of referrer spam.

        -Andy

        Kokolo 1 Reply Last reply Reply Quote 1
        • Kokolo
          Kokolo @Andy.Drinkwater last edited by

          Nono, I haven't had any contact with this website before. I've seen it only as crawl issue, I see it on Moz Crawl Report Too.

          Andy.Drinkwater 1 Reply Last reply Reply Quote 0
          • Andy.Drinkwater
            Andy.Drinkwater @Kokolo last edited by

            I haven't actually seen his issue before. It looks like your searches are producing an element in your RSS feed and I suspect this is being done to try and gain backlinks from you or is referrer spam.

            Add the following to your robots.txt. It can do no harm.

            disallow: /iseecars.com

            Someone else might pop along with a little more advice on this but I suspect it is something automated.

            -Andy

            Kokolo 1 Reply Last reply Reply Quote 2
            • Kokolo
              Kokolo @Andy.Drinkwater last edited by

              Well it's weird, I went to this one article that Moz was pointing out, and I did saw a link to ISeeCars.com, which was added by my contributors probablt.

              The thing is, the link is directing to Iseecars.com, but when you click on it, It redirects you to:

              mysite.com/iseecars.com

              I don't know why?!

              When I edited the post, I double checked that the link is linking only to ISEECARS...

              ...the issue remained, so I gave up and deleted the link.

              I'll wait another week for crawl reports.

              Thanks for the guidance though,

              Cheers,

              Andy.Drinkwater 1 Reply Last reply Reply Quote 0
              • Andy.Drinkwater
                Andy.Drinkwater @Kokolo last edited by

                Do update the thread when you have more data Nikola. I would be interested to see what else is found.

                -Andy

                1 Reply Last reply Reply Quote 3
                • Chenzo
                  Chenzo last edited by

                  Nikola, are you using wordpress ?

                  What is the URL?

                  Web therapist,

                  Chenzo

                  1 Reply Last reply Reply Quote 0
                  • 1 / 1
                  • First post
                    Last post
                  • Has anyone ever tested to see if having an ads.txt file provided any SEO lift?
                    scodtt
                    scodtt
                    0
                    3
                    48

                  • Reason for robots.txt file blocking products on category pages?
                    Frankie-BTDublin
                    Frankie-BTDublin
                    0
                    7
                    183

                  • Block parent folder in robot.txt, but not children
                    Martijn_Scheijbeler
                    Martijn_Scheijbeler
                    0
                    3
                    547

                  • Robots.txt being blocked
                    SOM24
                    SOM24
                    0
                    9
                    1.4k

                  • Losing indexed pages on Bing & Google - robots.txt
                    Tustep
                    Tustep
                    0
                    5
                    1.6k

                  • Search directory - How to apply robots
                    Blenny
                    Blenny
                    0
                    3
                    276

                  • What are the 10 commandments of web design?
                    sarenausa
                    sarenausa
                    0
                    4
                    1.1k

                  • Correct use for Robots.txt
                    EricVallee34
                    EricVallee34
                    0
                    5
                    1.0k

                  Get started with Moz Pro!

                  Unlock the power of advanced SEO tools and data-driven insights.

                  Start my free trial
                  Products
                  • Moz Pro
                  • Moz Local
                  • Moz API
                  • Moz Data
                  • STAT
                  • Product Updates
                  Moz Solutions
                  • SMB Solutions
                  • Agency Solutions
                  • Enterprise Solutions
                  • Digital Marketers
                  Free SEO Tools
                  • Domain Authority Checker
                  • Link Explorer
                  • Keyword Explorer
                  • Competitive Research
                  • Brand Authority Checker
                  • Local Citation Checker
                  • MozBar Extension
                  • MozCast
                  Resources
                  • Blog
                  • SEO Learning Center
                  • Help Hub
                  • Beginner's Guide to SEO
                  • How-to Guides
                  • Moz Academy
                  • API Docs
                  About Moz
                  • About
                  • Team
                  • Careers
                  • Contact
                  Why Moz
                  • Case Studies
                  • Testimonials
                  Get Involved
                  • Become an Affiliate
                  • MozCon
                  • Webinars
                  • Practical Marketer Series
                  • MozPod
                  Connect with us

                  Contact the Help team

                  Join our newsletter
                  Moz logo
                  © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                  • Accessibility
                  • Terms of Use
                  • Privacy