Walk me through how you would design and analyze an A/B test for a business initiative. What are the key considerations for sample size, duration, and statistical validity?

Sample answer preview

Designing a rigorous A/B test requires careful planning before any data is collected. A poorly designed test wastes time and can lead to confidently wrong conclusions, so getting the setup right is essential. Start by defining the hypothesis and success metric clearly.