How To A/B Test Videos For Higher Conversion Rates

For Royal Navy sailors in the 18th century, disease during long sea voyages was often a bigger threat than any enemy.

Scurvy (an illness that if left untreated can lead to a slow and painful death) was rife on ships in the 1700s.

And while many claimed to know of remedies that aided sufferers, no one came close to a cure until 1747 when Mr. James Lind, a Royal Navy ship surgeon, conducted what could be regarded as one of the very first A/B tests to find a cure.

He took 12 men suffering from scurvy and treated them with varying remedies. Some men were told to drink a quart of cider a day, others were prescribed two spoonfuls of vinegar three times a day, or two oranges and one lemon per day.

By the end of the first week of testing, the men given citrus fruits were well enough to care for the others. Thanks to A/B testing, a cure was found!

Nowadays, A/B testing feels as though it has been claimed by website designers who want to know which landing page looks prettiest.

But it is clear from the example above that A/B testing can help solve all kinds of problems…even when it comes to video content!

If you think videos are just a ‘set it and forget it’ kind of medium, then you could be losing out on tons of views, sign-ups, sales–or any other result you want from your videos. Just like with any form of content, you need to measure your results to improve and optimize your future marketing efforts.

In this article, we’re going to take a look at how A/B testing works, the rise of video content, and most importantly, how A/B testing can help with your video content.

How A/B Testing Works…

In the example above, James Lind tested 12 different men with a variety of different remedies, but A/B testing as we know it today is usually much simpler than that.

The term A/B refers to just two versions: version A and version B.

The idea is that you deploy two different versions of your video, half of your audience will see version A and half of your audience will see version B.

From there, you measure the results to determine which version performed better. Then you find a winner and roll that out to the masses.

A/B testing is important because it allows you to find out what your viewers really think of your video content.

It’s easy to look at stats and data from other sources and determine that viewers like upbeat, snappy videos under 60 seconds–but that may not be the case for the people watching your videos. You should never assume.

Video hosting site, Wistia assumed that their viewers would like to see a shorter onboarding video when joining the site as a new user and thanks to A/B testing, they found that assumption to be incorrect.

When they realized their current onboarding video was only generating an 18% play rate, Wistia thought that the length of the video (5 minutes) was off-putting to viewers.

So they made a new 2-minute video and hypothesized that this change would increase the percentage of new users uploading videos by approximately 10%:

In reality, the new video actually decreased video uploads by 3%:

 

Thanks to A/B testing, Wistia was able to catch this early and revert back to their old video.

Instead, they edited their original video down to 3 minutes and this helped them achieve the results they wanted.

Of course, length is only one aspect of video content that you could test.

You could also experiment with varieties in background colors, thumbnails, presenters or voiceover artists, animated or live action…the list is essentially endless!

The most important factor to consider when it comes to A/B testing is the data. Ask yourself: What is my goal?

If your goal is to increase sign-ups, and the video with the blue background resulted in 15 sign ups and the video with the red background resulted in 5 sign-ups, then the blue one wins.

The same applies no matter what your goal is.

It’s important to also remember that A/B testing doesn’t have to be a one-stop experiment.

You can optimize as many times as you want to in order to achieve the results you need.

For example, take the video with the blue background, make a version with a female voiceover and a male voiceover, and test again.

If the male voiceover gets 20 sign-ups and the female voiceover results in 30, you have a video that is now performing twice as well as the original blue background video, and 6x better than the red background one!

The Rise of Video Content

The first video was posted on YouTube in 2005, almost fifteen years ago! It’s a 20-second clip titled ‘me at the zoo’. Check it out below if you’re not one of the 57,000,000 people who have seen it:

 

Ever since then, YouTube has grown, and grown, and grown! Nowadays, approximately 300 hours of video are uploaded to YouTube every single minute!

And it’s not just YouTube, video is everywhere. By 2019, internet video traffic will account for 80% of all consumer internet traffic.

But just because video is popular and enjoyed by people all over the world, it doesn’t make it a surefire path to success for your business.

In fact, with every hour that passes (and another 300 hours of video are uploaded to YouTube) it becomes more competitive, and therefore more difficult to get the results you want from your videos.

That’s where A/B testing comes in. With A/B testing, you can refine and optimize your videos just as you would any other aspect of your marketing in order to achieve the best possible results.

A/B testing is a great way to do this and only recently has this become a realistic possibility.

How to A/B Test Your Videos

So A/B testing sounds pretty great in theory, but how can you put it into practice?

The first step is to make two (or more) versions of your video. If you’re struggling to think of how to make your videos different, here’s a list of factors you could change:

• Video player color
• Voiceover artist/Presenter
• Thumbnail
• Length
• Calls to action

After that, you need to decide what you want the goal of your video to be. Here are a couple of ideas:

• Increase sign-ups
• Views
• Conversions
• Brand awareness
• Increase user understanding

Next, you’re ready to test the impact of your video content against the metrics you want to measure. You just need to roll out the different versions to different viewers. There are a couple of different ways you can do this, and the method you choose will depend on what you’re trying to achieve. Let’s take a look at how to A/B test your videos with three different tools…

1. TubeBuddy

YouTube currently doesn’t offer any A/B testing tools or functionality, but TubeBuddy is an extension that can be used to help you better understand the performance of your video content on YouTube.

One of the many features of TubeBuddy is A/B testing:

The tool allows you to select two variations of the same video to find out which thumbnail, tags, description, and title perform best.

Over a two week period, TubeBuddy will then automatically rotate your videos and gather data.

At the end of the 2-week test, TubeBuddy presents you with data to show you which video got more views, which got more engagement, and where traffic came from for both:

They also give you an easy-to-understand ‘score’ that tells you exactly which video performed best, and by how much:

COST: To access the A/B testing tool, you need to pay for the Legend package which is currently $49 per month.

2. Facebook Ads

If you want to A/B test your videos on Facebook to see which one would perform best, you can use Facebook Ads Manager to run two variants of the same video.

It’s not as complicated as it sounds. All you need to do is set up a video ad as normal.

Answer the first question about your marketing objective and then scroll down and select the ‘split test’ toggle (this is just another term for A/B test):

You will then be asked what you want to measure, and from there you can create your variables. As you can see from the below example, we decided to aim Ad Set A at 18-25-year-old males and Ad Set B at 18-25-year-old females:

The only downside of creating an A/B test on Facebook is that you need to set aside a budget for it.

Not only that, you’re essentially paying for two ads. But the insight you will gain should help you focus your video ads in the future to make them more effective.

COST: The great thing about Facebook is that you can pick a daily budget that you’re comfortable with.

Of course, the higher your budget, the more data you will have to analyze. As you can see in the above screenshot, Facebook suggests £200 per day to A/B test videos (with a daily budget of £100 per video).

3. Wistia A/B Testing

Wistia has just released a new A/B testing feature for users in their advanced plan. This tool makes A/B testing very easy as all you need to do is select two videos to compare and Wistia does the rest.

They generate a special embed code that ensures they display a 50/50 split of your videos to viewers.

All you need to do is navigate to the stats page every now and then to check up on your current A/B tests, create new ones, and even review archived ones.

COST: You need to get in touch with Wistia to discuss pricing for their Advanced package.

Their Pro package is $99 per month, so we can hazard a guess that it will be more expensive than this.

If you don’t have the budget to spend on one of the above tools, there is another way you can A/B test your videos.

Google Analytics is a free tool that allows you to measure lots of different aspects of your website to track and improve its performance.

You can use this to A/B test the videos on your website by creating two different versions of the same webpage–with the only difference being one will show video A and one will show video B.

To do this, head to Google Analytics and click on ‘experiments’ under the Behavior tab:

You can then create an experiment to find out which variant of your webpage performs best:

This may seem like a convoluted way to simply test your videos, but as A/B testing for videos is relatively new and expensive this is a great way to get the same result for free.

Whichever method or tool you use, the important thing to remember is that you want your test to be as random as possible.

The only way to gain truly insightful knowledge from your A/B tests is to ensure that users don’t know a test is happening.

After all, if James Lind had told his scurvy sufferers that vinegar was definitely going to cure them, he might have seen vastly different results thanks to the ‘placebo’ effect.

Thanks for Reading

Video content has become one of the main sources for how we find, digest and share information with the world–and the presence of videos online is only set to grow in the future.

To ensure that your video content stands out among the crowd and achieves the results you want it to, you need to make sure that it is the best it can be, every time.

Just as Mr. James Lind used A/B testing to cure scurvy all those years ago, you can use it to cure any poorly performing video content!

This article was written by Charlie Carpenter.

Charlie Carpenter is the co-founder and CEO of Kite. He is a mobile advocate with over ten years of industry experience.

After working for large and small agencies for many years, he co-founded Kite; a software solution for print-on-demand, zero inventory merchandise, and personalized photo print goods. As well as an entrepreneur, Charlie is a seasoned product strategist with experience of various types of digital projects which include: Responsive and Adaptive Websites, Mobile & Tablet Apps, Hybrid Apps, Cross-Platform App development.

You can connect with Charlie on LinkedIn, and follow him on Twitter.