Regular Expressions for Filtering BOT Traffic?

[SErOb]

"...a combination of operating system, location, and some other factors can do the trick."

Yep, combined with those, look for "Avg. Time on Page = 00:00:00"

[Chris_CM]

Time on page can be a tricky one because sometimes actual visits can record 00:00:00 due to the way it is measured.  I'd recommend using other factors like the ones I mentioned above.

[AWCthreads]

For example,

Since I implemented the filter four days ago, rackspace cloud servers have visited my site 848 times, , visited 1 page each time, spent 0 seconds on the page and bounced 100% of the time.

What is the reg expression for rackspace?

[Chris_CM]

How is it titled in the ISP report exactly?

[AWCthreads]

rackspace cloud servers

Maybe my problem is I'm not looking in the right place.

I'm in audience>technology>network and the column shows "service provider."

[SErOb]

Agreed! That's why I suggest using it in combination with the variables you mentioned above.

[Chris_CM]

Ok, try this:

^(microsoft corp|inktomi corporation|yahoo! inc.|google inc.|stumbleupon inc.|rackspace cloud servers)$|gomez

Just added rackspace as another match, it should work if the name is exactly right.

Hope this helps,

Chris

[AWCthreads]

Does it need the . before the )

[Chris_CM]

Not unless there's a . after the word servers in the name.  The . is escaping the . at the end of stumbleupon inc.

[AWCthreads]

Crap.

Well, I guess the vernacular is what I need to know.

Knowing what to put where is the trick isn't it? Is there a dummies guide somewhere that spells this out in kindergarten speak?

I could really see myself botching this filtering business.

[Chris_CM]

If you copy and paste my RegEx, it will filter out the rackspace bots.  If you want to learn more about Regular Expressions, here is a site that explains them very well, though it may not be quite kindergarten speak.

[AWCthreads]

I will definitely do that for Rackspace bots, Chris.

Thank you for taking the time to walk me through this and tweak my filter.

I'll give the site you posted a visit.

[Chris_CM]

No problem, feel free to reach out if you have any other RegEx related questions.

Regards,

Chris