The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Regular Expressions for Filtering BOT Traffic?

    Regular Expressions for Filtering BOT Traffic?

    Intermediate & Advanced SEO
    17 3 1.9k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Chris_CM
      Chris_CM @AWCthreads last edited by

      How is it titled in the ISP report exactly?

      AWCthreads 1 Reply Last reply Reply Quote 0
      • AWCthreads
        AWCthreads @Chris_CM last edited by

        rackspace cloud servers

        Maybe my problem is I'm not looking in the right place.

        I'm in audience>technology>network and the column shows "service provider."

        Chris_CM 1 Reply Last reply Reply Quote 0
        • SErOb
          SErOb @Chris_CM last edited by

          Agreed! That's why I suggest using it in combination with the variables you mentioned above.

          1 Reply Last reply Reply Quote 0
          • Chris_CM
            Chris_CM @AWCthreads last edited by

            Ok, try this:

            ^(microsoft corp|inktomi corporation|yahoo! inc.|google inc.|stumbleupon inc.|rackspace cloud servers)$|gomez

            Just added rackspace as another match, it should work if the name is exactly right.

            Hope this helps,

            Chris

            AWCthreads 1 Reply Last reply Reply Quote 1
            • AWCthreads
              AWCthreads @Chris_CM last edited by

              Does it need the . before the )

              1 Reply Last reply Reply Quote 0
              • Chris_CM
                Chris_CM last edited by

                Not unless there's a . after the word servers in the name.  The . is escaping the . at the end of stumbleupon inc.

                AWCthreads 1 Reply Last reply Reply Quote 1
                • AWCthreads
                  AWCthreads @Chris_CM last edited by

                  Crap.

                  Well, I guess the vernacular is what I need to know.

                  Knowing what to put where is the trick isn't it? Is there a dummies guide somewhere that spells this out in kindergarten speak?

                  I could really see myself botching this filtering business.

                  Chris_CM 1 Reply Last reply Reply Quote 0
                  • Chris_CM
                    Chris_CM @AWCthreads last edited by

                    If you copy and paste my RegEx, it will filter out the rackspace bots.  If you want to learn more about Regular Expressions, here is a site that explains them very well, though it may not be quite kindergarten speak.

                    AWCthreads 1 Reply Last reply Reply Quote 1
                    • AWCthreads
                      AWCthreads @Chris_CM last edited by

                      I will definitely do that for Rackspace bots, Chris.

                      Thank you for taking the time to walk me through this and tweak my filter.

                      I'll give the site you posted a visit.

                      Chris_CM 1 Reply Last reply Reply Quote 0
                      • Chris_CM
                        Chris_CM @AWCthreads last edited by

                        No problem, feel free to reach out if you have any other RegEx related questions.

                        Regards,

                        Chris

                        1 Reply Last reply Reply Quote 1
                        • 1 / 1
                        • First post
                          Last post
                        • Website Traffic Is Down
                          dohertyjf
                          dohertyjf
                          0
                          2
                          108

                        • How do I get the sub-domain traffic to count as sub-directory traffic without moving off of WordPress?
                          Chris_Hickman
                          Chris_Hickman
                          0
                          5
                          67

                        • Blog Traffic
                          MoosaHemani
                          MoosaHemani
                          0
                          3
                          124

                        • Traffic dropped suddenly
                          KeriMorgret
                          KeriMorgret
                          0
                          2
                          92

                        • Almost no organic traffic
                          Dr-Pete
                          Dr-Pete
                          0
                          6
                          240

                        • Regular Expression / Wildcard Redirect Situation
                          MikeTek
                          MikeTek
                          0
                          2
                          376

                        • How to increase the traffic ?
                          NeSEO
                          NeSEO
                          0
                          6
                          473

                        • Traffic from twitter
                          seoug_2005
                          seoug_2005
                          0
                          5
                          637

                        Get started with Moz Pro!

                        Unlock the power of advanced SEO tools and data-driven insights.

                        Start my free trial
                        Products
                        • Moz Pro
                        • Moz Local
                        • Moz API
                        • Moz Data
                        • STAT
                        • Product Updates
                        Moz Solutions
                        • SMB Solutions
                        • Agency Solutions
                        • Enterprise Solutions
                        • Digital Marketers
                        Free SEO Tools
                        • Domain Authority Checker
                        • Link Explorer
                        • Keyword Explorer
                        • Competitive Research
                        • Brand Authority Checker
                        • Local Citation Checker
                        • MozBar Extension
                        • MozCast
                        Resources
                        • Blog
                        • SEO Learning Center
                        • Help Hub
                        • Beginner's Guide to SEO
                        • How-to Guides
                        • Moz Academy
                        • API Docs
                        About Moz
                        • About
                        • Team
                        • Careers
                        • Contact
                        Why Moz
                        • Case Studies
                        • Testimonials
                        Get Involved
                        • Become an Affiliate
                        • MozCon
                        • Webinars
                        • Practical Marketer Series
                        • MozPod
                        Connect with us

                        Contact the Help team

                        Join our newsletter
                        Moz logo
                        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                        • Accessibility
                        • Terms of Use
                        • Privacy