Visual Website Optimizer vs. Optimizely vs. Conductrics

[SteveTheWoodsman]

I've always done my own A/B testing and haven't used a software package.  I've heard a lot of good about Visual Website Optimizer.  However, I like that Conductrics uses statistical modeling where the others don't. So what do you see as being the really big pluses?

Note: I need to find a platform so that we can do testing faster and for more clients at a time... I just don't have the time to hand build and track each client's multivariation test. Thus the need for software.  Therefore, ease of use, quick integration, and good analytics are important in my decision making process.

Thanks in advance!

[willcritchlow]

I haven't used Conductrics in anger, but have recently started using Optimizely more - I really like the fact that it's all jQuery based. The big benefits of that for us are:

• Super easy point and click interface for non-technical writers to make changes
• More advanced (edit jQuery code) options for more involved tests
• Not limited to page snapshots (as I believe VWO is) - this was something that caused us issues with our XSS protection scripts and meant we couldn't use VWO effectively on distilled.net

Hope that helps - sorry I haven't got comprehensive comparison data. Optimizely has a free trial though so you can test it out for yourself (really quick to get up and running).

[SteveTheWoodsman]

Will,

Thanks for the awesome feedback - especially about the page snapshots..

"I haven't used Conductrics in anger" - not sure that you meant anger or ages but either way I would be interested to know why you haven't used it?  I like the idea behind machine learning and real-time implementation, but I haven't found a lot of people that have much to say about it.  On the flipside, I'm always a little nervous that the machine learning process will leave out key information that could only be noticed by a human and therefore optimize incorrectly - thus, I lean more towards Optimizely/VWO.

Thanks again!

Steve

[willcritchlow]

I meant anger - just short-hand for "haven't used it on anything live". I've played around with demos etc but haven't used it personally on live projects.

I believe they're doing what's called bandit optimization in the statistical literature. Fascinating area of study. I like it from a mathematical perspective - though I prefer declaring a specific winner and then implementing that version rather than always having the split-test software running and serving up variations.

It feels tidier, easier to maintain and results in a faster website...