Getting the right people to a website or application is one challenge, but getting them to take the actions that you want to on the product is another. CRO is a process whereby tests are run on a product in order to improve the conversion rates to desired behaviour on the product..
After the dot com bubble burst in the early 2000s ecommerce sites needed to find new ways to increase the return on their marketing activities, and get more performance from the products they had created. CRO testing was born, where a growth marketing team (or likely a development team back then) amended elements of a product or campaign landing page such as text on, colour of or position of a button and measure the results against a control version. This means that a team can optimise performance iteratively, making minor performance gains all the way down a product funnel, which compound over time.
Statistics play a significant role in CRO testing, because not all customer behaviour is consistent. Traffic and user intent can fluctuate from hour to hour, and minute to minute. This creates ‘noise’ in the data, and a range of statistical methodologies exist (Bayesian and Frequentist) to calculate a test’s statistical significance. This statistical significance, often referred to as the P-value, is effectively the risk that the results are wrong. The closer to 1 this number becomes the lower the risk the results are wrong. However, it should be noted; what statistical methodology you use to calculate your P-Value will depend on the test itself. If you have ‘small data’ and previous experience (or an expectation) - often referred to as a prior - then use Bayesian as you will reach significance more quickly. If you have larger data sets Frequentist would be the weapon of choice. But there is a caveat in that each test designed will have to evaluate what methodology to use, and the rule above it not 100% consistent.
A range of 3rd party platforms exist enabling a business to run CRO tests across both desktop/mobile websites and applications providing both the technology and statistical calculations required to run testing.
But those running CRO testing should be very mindful of test design, sample sizes and variations such as device type, day of the week, traffic source (alongside a myriad of other potential variables) to inform their results. If done badly, CRO can easily ‘de-optimise’ a product/website making test design the most important aspect of a CRO team’s workload.
The design and implementation of a test should be as scientific as possible and include the following elements:
CRO should be a major focus for all growth marketing teams, as any performance gains you achieve here will increase the ROI on all other growth marketing activities, and compund over time.
Key areas of CRO include: