Compatibility Testing for Conversion Experiments

Compatibility

Would you continue browsing a website that is either broken beyond repair or looks terrible on your device? Your visitors won’t either. For every test that you publish you have to make sure all variations work properly, without any bugs or user interface problems.

Target audience

Unfortunately it won’t be possible to test your variations on every possible combination of device, browser and screen size. Therefore it’s important to know who your target audience is, and what technology they are using. This way you can optimize for these technologies and test those for compatibility with your testing variations.

What technology are they using?

The easiest way to find out what devices, operating systems, browsers and screen sizes your visitors are using is Google Analytics. Open your default view, head to the ‘Audience’ section, then ‘Technology’ > ‘Browser & OS’. You can now use the ‘Primary dimension’ links to select the data you are looking for.

Browser

The most common browsers on desktop devices are Google Chrome, Firefox and Internet Explorer (IE). Lagging behind are Safari and Opera. However, the latter two browsers have major market shares on mobile devices. I would advise you to run the latest version of as many as possible of these browsers natively on your computer.

IE for Mac users

For Mac users running IE unfortunately is a bit more difficult. Aside from several commercial solutions, you might consider running IE under a VirtualBox or if you could use an online tool such as BrowserStack to take screenshots.

Device and Screen size

devices In order to truly see what you visitors are seeing, you need to be able to use (or emulate) their devices and screen sizes as well. While it’s still preferably to have a set of physical devices to test on, emulators are becoming more powerful and easy to use as well. For example, Google Chrome now comes with Device Mode & Mobile Emulation that let’s you easily switch between devices and corresponding screen sizes. Of course, you can emulate different screen sizes for desktop devices by simply dragging the browser window to a small size.

Operating system, geography, language

While these are less likely than the factors above to cause any problems, they need to be checked for compatibility anyways. For example, Windows often renders fonts in a different size than Mac computers do. Also, some websites have specific settings or personalized content for different geographies or languages. Be sure to check up with other departments whether that is the case for your website as well.


Dutch translation: Compatibiliteit Nakijken voor Conversie Experimenten


Theo van der Zee

Author: Theo van der Zee

He is the founder of ConversionReview. He has been building and optimizing websites for 15+ years now, and doing so with great success.

On top of his digital skills, Theo is also a trained psychologist and frequent speaker at events around the world.