Class 13 Case Study: Improve User Engagement for Instagram Using A/B/N Testing

Author

Affiliation

Dr Wei Miao

UCL School of Management

Published

November 13, 2024

1 Case Background

1.1 Business Objective

Instagram aims to increase user engagement and activity.
We can propose gamification strategies based on scientific theories.
Need to empirically test whether proposed gamification strategies are effective using A/B/N testings.

1.2 Situation Analysis

Conduct a situation analysis to assess Instagram’s business environment in the UK:¹

What is Instagram’s business model?
How does Instagram make revenues?
Who are Instagram’s customers?
What are the major competitors and their relative strengths and weaknesses compared with Instagram?
Who are the collaborators of Instagram?
PESTLE analysis: any particular legal and regulatory issues that Instagram needs to be aware of?

2 Theoretical Motivations

2.1 Theoretical Motivation for Business Ideas

When proposing business ideas, we should base our proposals on scientific, well-established theories from different disciplines.
- Bottom-up approach: start with theories and then generate business ideas
- Top-down approach: start with business ideas and then find theories to support them
Let’s first see some examples of behavioral economics theories!

2.2 Behavioral Theories

2.3 Default Effect

Default effect

2.4 Left-Digit Bias

Left-digit bias

2.6 Prospect Theory

Prospect theory posits that people feel more pain from losing something than pleasure from gaining something.
This theory can be used to explain why people are more likely to engage in activities that prevent loss than those that promote gain.

2.7 Business Proposal

Implement gamification features on Instagram to increase user activity based on the theories of Social Comparison and Prospect Theory.
Let’s think about ideas that can boost user engagement on Instagram.

2.8 Potential Strategies

Endowment effect: Implementing a points and badge system to create sense of ownership and encourage engagement (e.g., likes, comments, shares).
Social comparison theory: Leaderboards showing top users; Social comparison through activity rankings
Prospect theory: Time-limited rewards and achievements

3 A/B/N Testing for Instagram

3.1 Step 1: Decide on the Unit of Randomization

What would be the best unit of randomization?

3.2 Step 2: Mitigate Spillover and Crossover Effects

What are the potential problems for spillover and crossover?

3.3 Step 3: Decide on Randomization Allocation Scheme

How should we determine the randomization scheme?

3.4 Step 4: Collect Data

What is the sample size we need?
What data should we collect?

3.5 Step 5: Data analytics

Randomization checks
How to estimate the treatment effects?

3.6 After-Class

(optional) Test and learn: How a culture of experimentation can help grow your business

Footnotes

Suggested answers are on the course website.↩︎

--- date: "`r (lubridate::ymd('20241002')+lubridate::dweeks(6))`" title: "Class 13 Case Study: Improve User Engagement for Instagram Using A/B/N Testing" --- # Case Background ## Business Objective - Instagram aims to increase user engagement and activity. - We can propose gamification strategies based on scientific theories. - Need to empirically test whether proposed gamification strategies are effective using A/B/N testings. ## Situation Analysis Conduct a situation analysis to assess Instagram’s business environment in the UK:^[Suggested answers are on the course website.] - What is Instagram's business model? - How does Instagram make revenues? - Who are Instagram's customers? - What are the major competitors and their relative strengths and weaknesses compared with Instagram? - Who are the collaborators of Instagram? - PESTLE analysis: any particular legal and regulatory issues that Instagram needs to be aware of? # Theoretical Motivations ## Theoretical Motivation for Business Ideas - When proposing business ideas, we should base our proposals on scientific, well-established theories from different disciplines. - **Bottom-up approach**: start with theories and then generate business ideas - **Top-down approach**: start with business ideas and then find theories to support them - Let's first see some examples of behavioral economics theories! ## Behavioral Theories - [Framing effect](https://www.investopedia.com/framing-effect-7371439) - [Endowment effect](https://www.investopedia.com/terms/e/endowment-effect.asp) - [Mental accounting](https://www.investopedia.com/terms/m/mentalaccounting.asp) ## Default Effect - [Default effect](https://en.wikipedia.org/wiki/Default_effect) ::: {.columns} ::: {.column width='50%'} ```{r} #| echo: false #| fig-align: 'center' knitr::include_graphics("images/Week 7/organ donation.png") ``` ::: ::: {.column width='50%'} ```{r} #| echo: false #| fig-align: 'center' knitr::include_graphics("images/Week 7/default effect.png") ``` ::: ::: ## Left-Digit Bias - [Left-digit bias](https://journals.sagepub.com/doi/full/10.1177/0022243720932532) ::: {.columns} ::: {.column width='50%'} ```{r} #| echo: false #| fig-align: 'center' knitr::include_graphics("images/Week 7/iPhone pricing.png") ``` ::: ::: {.column width='50%'} ```{r} #| echo: false #| fig-align: 'center' knitr::include_graphics("images/Week 7/left digit bias.jpg") ``` ::: ::: ## Social Comparison Theory - People evaluate their own opinions and abilities by comparing themselves to others, especially when comparing oneself to similar others. - Social comparison can be upward or downward. - Social comparison can motivate people to improve their performance; however, it can also lead to negative emotions. ```{r} #| echo: false #| fig-align: 'center' knitr::include_graphics("images/Week 7/Social Comparison Theory.jpeg") ``` ## Prospect Theory - Prospect theory posits that people feel more pain from losing something than pleasure from gaining something. - This theory can be used to explain why people are more likely to engage in activities that prevent loss than those that promote gain. ```{r} #| echo: false #| fig-align: 'center' knitr::include_graphics("images/Week 7/Prospect Theory 2.png") ``` ## Business Proposal - Implement gamification features on Instagram to increase user activity based on the theories of Social Comparison and Prospect Theory. - Let's think about ideas that can boost user engagement on Instagram. ## Potential Strategies - **Endowment effect**: Implementing a points and badge system to create sense of ownership and encourage engagement (e.g., likes, comments, shares). - **Social comparison theory**: Leaderboards showing top users; Social comparison through activity rankings - **Prospect theory**: Time-limited rewards and achievements # A/B/N Testing for Instagram ## Step 1: Decide on the Unit of Randomization - What would be the best unit of randomization? ## Step 2: Mitigate Spillover and Crossover Effects - What are the potential problems for spillover and crossover? ## Step 3: Decide on Randomization Allocation Scheme - How should we determine the randomization scheme? ## Step 4: Collect Data - What is the sample size we need? - What data should we collect? ## Step 5: Data analytics - Randomization checks - How to estimate the treatment effects? ## After-Class - (optional) [Test and learn: How a culture of experimentation can help grow your business](https://www.thinkwithgoogle.com/intl/en-gb/marketing-strategies/data-and-measurement/test-learn-experiment-culture/)