Campaign Analysis (A/B Testing)

Goal

Drive revenue growth and improve campaign outcomes by leveraging in-depth analysis of ad performance to refine ad content, targeting, or placement, based on quantifiable impacts on key performance metrics.

Result

Inform marketing budget allocation through comprehensive evaluation of campaign results, providing actionable insights for optimizing future ad campaigns.

Case

To evaluate the influence of website advertisements on sales, a large company conducted a 31-day A/B testing campaign. By randomly assigning 20,000 customers to different groups, the company aimed to determine the causal relationship between ads and sales performance.

Was the Campaign Successful?

Yes, statistical analysis using the Chi-squared test reveals a rejection of the Null hypothesis, as the calculated p-value falls below the predetermined significance level. Additionally, the Chi-statistic value provides evidence of the advertisement experimental group's superior performance.

Which Days Were the Best for Advertising?

During days 19-24, the conversion rate experienced the most significant boost, while showing comparatively lower performance in the initial days of the month.

At What Time of the Day Were Ads More Successful?

The strongest uplift, in comparison to the control group, is noticeable during the early morning. More specifically, between midnight and 4:00 am. It can be seen too, that the performance is more consistently in the experimental group.

What Were the Optimal Numbers of Ads to Show?

Drawing from the information at hand, a logical inference would be that 5 ads represent the optimal number. A reduction in the number of ads would lead to a significant decrease in customers' inclination to make a purchase, while an increase beyond this threshold would likely annoy them.